Real-time modelling and forecasting

How can we better prepare
for the next pandemic?


Sebastian Funk (@sbfnk)
https://epiforecasts.io

14 June, 2022
BIRS-UBCO Workshop on "Preparing for the next pandemic"
Banff International Research Station

Mechanistic models support causal understanding, but predictions can have value in their own right

Desai et al., Health Secur, 2019
Keeling et al., Stat Meth Med Res, 2021
Cramer et al., medRxiv, 2021

Real-time forecasts can inform decision making

Anticipate healthcare demand

Real-time forecasts can inform decision making

Inform interventions

Real-time forecasts can inform decision making

Support clinical trials

Global COVID case forecasts via the renewal equation

\[I(t) = R_t \sum_{\tau} g_{\tau} I_{t-\tau}\]

Abbott et al., Wellcome Open Res, 2020
Gostic et al., PLoS Comp Biol, 2021
https://epiforecasts.io/posts/2022-03-25-rt-reflections

Forecasting to inform policy in the UK

Funk et al., medRxiv, 2021

COVID forecast hubs

Reich et al., Am J Public Health, 2022
https://covid19forecasthub.org

  1. How good were COVID forecasts?
  2. How could COVID forecasts be improved?
  3. What can we conclude for the next pandemic?

How good were COVID forecasts?

We can compare forecasts using
proper scoring rules, e.g. \[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\]

Forecast hubs:
median ensemble outperforms individual models

Sherratt et al., in prep

We can compare forecasts using proper scoring rules, e.g. \[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\] but these only tell us about relative quality of forecasts

Absolute quality of forecasts #1: baseline models

Absolute quality of forecasts #2: human vs. machine

Bosse et al., medRxiv, 2022

Absolute quality of forecasts #3: calibration

Sherratt et al., in prep

How could COVID forecasts be improved?

Including possible predictors #1: cases

Meakin et al., BMC Medicine, 2021

Including possible predictors #2: contacts

Munday et al., in prep

Including possible predictors #3: variants

https://github.com/epiforecasts/forecast.vocs
https://github.com/jbracher/branching_process_delta

What can we conclude for the next pandemic?

Summary

  • Outbreak forecasting is difficult
  • Ensembles perform best, but can be difficult to interpret

Forecasting and nowcasting will remain relevant

UKHSA, 2022

Nowcasting of delayed reports

https://covid19nowcasthub.de
https://epiforecasts.io/epinowcast/

New initiatives

We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools

Acknowledgements

EpiForecasts group (https://epiforecasts.io)
Akira Endo, Hannah Choi, Hugo Gruson, James Munday,
Kath Sherratt, Nikos Bosse, Sam Abbott, Sophie Meakin

Johannes Bracher, Nick Reich and other collaborators